Transfer-Entropy-Regularized Markov Decision Processes

We consider the framework of transfer-entropy-regularized Markov decision process (TERMDP) in which weighted sum classical state-dependent cost and transfer entropy from state random to control input is minimized. Although TERMDPs are generally formulated as nonconvex optimization problems, an analytical necessary optimality condition can be expressed a finite set nonlinear equations, based on iterative forward–backward computational procedure similar Arimoto–Blahut algorithm developed. It shown that every limit point sequence generated by proposed stationary TERMDP. Applications discussed context networked systems theory nonequilibrium thermodynamics. The applied information-constrained maze navigation problem, whereby we study how price information qualitatively alters optimal polices.